The Fundamentals of Prioritising Requirements

In systems development projects the activity of prioritising requirements is carried out during the requirements engineering phase. The current trend is to develop systems that are quick-tomarket, bound by strict budget constraints, and released in progressive versions. The inherent risks with this trend include not adequately fulfilling the core requirements or simply missing important ones altogether. This research paper specifies the fundamental requirements for a prioritisation process. The research considers where prioritisation should take place during the requirements phase, and who should be involved in the prioritisation process. Current techniques such as AHP and QFD are analysed as to how well they satisfy the fundamental needs of a successful prioritisation process. A framework is described that incorporates the many aspects of prioritising requirements. These aspects include balancing the various viewpoints and goals of the stakeholders, determining the value of each stakeholder’s subjective opinion, and aligning the requirements to the business objectives of the systems development project. The Requirements Prioritisation Tool (RPT), which is based on the framework, is presented and compared with current methods. It is also evaluated with case studies for how well it satisfies the fundamental requirements of an efficient prioritisation process. INTRODUCTION TO PRIORITISING REQUIREMENTS Requirements prioritisation is the setting of ranks or ratings of importance to a set of requirements based on certain criteria like goals, risks, quality, use, and according to the viewpoints of various stakeholders. The notion of releasing progressive versions and updates on products, as well as the rising demand on developers to build systems that go to market much quicker than ever before has led to the need to prioritise requirements at the earliest possible stage in the systems development life cycle. Many development projects suffer because the most important requirements are not elicited and analysed at the early stages of the system development process. Meeting budgets and deadlines has become more important than designing and implementing an exhaustive yet comprehensive set of requirements. Most customers opt to reduce the number of product requirements that need to be fulfilled in order to meet their business goals. Prioritising requirements allows customers and developers to discuss and rate candidate requirements. It provides a means to solve conflicts and balance the expectations of the stakeholders for the proposed product. Prioritising requirements provides the necessary information to make negotiations and decisions on which requirements to discard or focus upon. It provides a means to measure agreement and disagreement amongst the stakeholders and help resolve which requirements are important. Prioritising requirements is an important activity in the requirements engineering lifecycle. It is a recursive activity that is used while the requirement analysts and the other stakeholders (customers, users) discuss and endeavour to reach agreement on which requirements to continue analyzing, modeling, and Systems Engineering, Test & Evaluation Conference, Sydney, Australia, October 2002 validating. Even when a set of requirements is agreed upon, prioritisation is still needed when requirements change. The volatility of requirements can be managed with a flexible prioritisation process in the requirements engineering life cycle. [1-6] FUNDAMENTAL REQUIREMENTS FOR A PRIORITISATION PROCESS In a prioritisation process, the following requirements should be considered: • The prioritisation process should avoid the risk of having biased or tainted stakeholder input affect the prioritisation process. • The prioritisation process should reduce the risk of missing important requirements • The prioritisation process should provide benefits that should outweigh the resources needed to apply the prioritisation process • The prioritisation process should have access to information from many different relevant viewpoints. • The prioritisation process should incorporate non-functional issues and business objectives in assigning priorities to the requirements. • The prioritisation process should support the evolution of the requirements into appropriate specifications for the system. • The prioritisation process should produce priorities that can be validated. There should be a rationale behind the assigned priorities for each priority. • The prioritisation process should be able to handle the volatility of a set of requirements. The prioritisation process should be able to cope with the varied levels of abstraction in requirements statements. The Who and When of Prioritising Requirements Prioritising requirements should involve representatives from each group of stakeholders with a vested interest in the success of the development project. Valuable input can be provided by all of the groups. The users can provide valuable information on functionality and user interfaces, whereas the managers can provide input on non-functional issues like security, and performance. The various levels of viewpoints can provide different insights on the same areas of functionality. The managers usually view the system from a much higher level than the users do. The various levels at which each stakeholder perceives and reasons about the system, can provide valuable input into why various functionalities may or may not be important. The scope of information will greatly improve the accuracy of the priorities. Trying to prioritise requirements from a completed specification document is quite time consuming and difficult for the stakeholders involved. There are usually hundreds and thousands of requirements that need to be rated. The costs and resources would outweigh any benefits the prioritisation process would provide. Requirements prioritisation should occur during the elicitation and analysis stages of the requirements engineering lifecycle when the number of requirements is still quite small and they are usually expressed in high-level terms. Early on in the requirements engineering process there is usually a small number of requirements that will guide the requirements analyst through to the elicitation of other detailed requirements. Prioritisation of the few high level requirements can be done without too much effort. The set of requirements can be viewed as a whole alleviating the problem of confusing the stakeholders with too many stimuli. Since the requirements at this early stage are usually expressed as high-level product features or functions, the interaction between the various groups of stakeholders can be done quite easily. The prioritisation of the high-level requirements can be done quite frequently as the stakeholders bounce back and forth ideas and try different combinations of requirements on the prioritisation process. The interaction between analysts and customers on the initial requirements using a versatile prioritisation process can prove very helpful in directing the analysis activities along the right track. Systems Engineering, Test & Evaluation Conference, Sydney, Australia, October 2002 Requirements prioritisation should be a progressive co-operative activity with the stakeholders. It shouldn’t be unilateral. Each stakeholder will have different priorities. Prioritising requirements should serve as a vital part of helping the stakeholders agree upon which requirements should be analysed and used to design and implement the desired system. CURRENT APPROACHES – HOW DO THEY SIZE UP? Some requirements engineers prefer to use Quality Function Deployment (QFD) [7] or the Analytical Hierarchy Process (AHP) [8] to prioritise requirements. Other approaches include the cost-value approach by Karlsson [9], Wiegers’ method [10] , as well as a multiplicity of ad hoc industrial practices such as team voting [11]. Quality Function Deployment. The QFD process consists of the Voice of the Customers or Customer Attributes, which are driven by the customer goals, the Voice of the Engineer or the Design Parameters, which are the technical measures needed to build the customer requirements, the Relationship Matrix, which describes the relationship between the customer requirements and the design parameters, and the Roof of the House of Quality (HOQ), which is the correlation between the design parameters [12]. Like any process, the quality of the output is only as good as the quality of the inputs and the controls over the process. With QFD it is critical to address the quality of the user requirements gathering methods, and the management of these requirements while building the house of quality. The QFD process consist of the following stages: • QFD Planning, • Requirements Gathering, • QFD Analysis (building HOQ) [13]. The size of the House of Quality is a crucial issue in QFD development projects. The suggested limit is to work with a 30 by 30 matrix. That is only 30 requirements, which makes QFD difficult to manage with large sets of requirements. Other concerns include: • The lack of measures to check the honesty of the subjective stakeholders inputs • The amount of tedious work needed to get the prioritisation results • The difficulty of changing the user requirements because of the amount of rework necessary. QFD relies on the user requirements being stable. The volatility of requirements is not really catered for. Another concern is the rapid transition from requirements analysis to design and technical issues. This rapid transition in QFD from user requirements to technical solutions does not allow for modeling and validation activities to be performed to the requirements. The jump to design issues can complicate the requirements analysis activities, and cause important requirements to be missed. The user requirements are not final and should not be treated as such. Requirements prioritisation should be trying to determine “what” the system should do, without referring to “how” it will do it [3]. There is no straightforward way of handling temporal relationships or dependencies among requirements in QFD. The user requirements are treated as atomic requirements, which is not realistic. Requirements are dependent on each other, and there are inter-relationships that need to be identified and used in the prioritisation process to produce accurate results. QFD works with ordinal scales of 1-5 and 1-9. There is substantial evidence to prove that ordinal scales are weak in accurately eliciting stakeholder preferences. This important issue of using scales is handled in section 4.1. Analytical Hierarchical Process AHP relies on the pair-wise comparisons of a set of requirements using a scale of 1-9. The steps of AHP include: • Structuring a hierarchy from the top (the objectives from a managerial viewpoint) through the intermediate levels (criteria on which subsequent levels depend) to the lowest levels Systems Engineering, Test & Evaluation Conference, Sydney, Australia, October 2002 (which usually is a list of the alternatives). • Construct a set of pair-wise comparison matrices for each of the lower levels. One matrix for each element in the level immediately above. A element in the higher level is said to be a governing element for those in the lower level. In fact, every element in the lower level affects every element in the upper level. The elements in the lower level are then compared to each other based on their effect on the governing element above. This yields a square matrix of judgments. The pair-wise comparisons are done on a scale of 1 to 9 showing which requirement is more important than the other and to what degree. (1=equal importance, 3=slighter more important, 5=essentially more important, 7=demonstrated importance, 9=extremely more important). If requirement A is slighter more important than B, then 3 is entered in row A, column B and the reciprocal is assigned to row B, column A. • There are n(n 1)/2 comparisons required to be made for each matrix, where n is the number of requirements used in the .matrix. • The consistency is determined using the eigenvalue [8, 14]. Expert ChoiceTM is a group metadecision support software product based on the decision-making methodology AHP. The Expert Choice method works by guiding decision-makers through a series of pair-wise comparisons to derive priorities for decision objectives and options. Expert Choice lets you combine and synthesize the judgments of any subset of decision-makers or the entire group to provide the full spectrum of different perceptions of a problem. AHP suffers from the problems caused by trying to model requirements into a dominance hierarchy or network. This can be quite a time-consuming and tedious exercise. AHP can only handle interrelationships of requirements under a higher element of the hierarchy. Similar to QFD, dependencies amongst requirements are not really handled as requirements can appear under many threads or concepts, and can’t realistically be restricted to such modelling techniques. Another concern is the restriction Saaty imposes on how many requirements should be used in a matrix. Saaty states that large subsets of requirements (anything over seven requirements to a set) results in computational explosions and is more prone to produce inconsistencies in the results [8]. There are no measures in place to check the motivation behind the stakeholders’ preferences, and changes in requirements result in (n-1) pair-wise comparisons that need to be redone for each new requirement. There aren’t any non-functional or business requirements used to help the stakeholders choose their preference, and there is no rationale provided for the preferences or results. Finally the use of a 1-9 scale causes complications for many stakeholders as stated in the case studies performed in [15] and by evidence presented in section 4.1. The use of a ninepoint scale including fractions has not been empirically proved to be optimal in system development projects. Karlsson’s Cost-Value Approach Karlsson has adopted parts of the AHP concept and developed a cost-value approach for prioritizing requirements. He compared the requirements in a pairwise manner with regard to their relative value of importance and relative cost to implement. The relative intensities that resulted were plotted on a cost-value diagram that revealed which requirements were better to use. He then improved the technique by reducing the number of comparisons needed by implementing local and global stopping rules [5]. The cost-value approach suffers from the same weaknesses as AHP. Psychologists [16-19] state that doing pairwise comparisons gives opportunity for deviation from strictly consistent rank order, since not all the stimuli are present for simultaneous observation and comparison. The use of a 1-9 scale, and the lack of handling requirements dependencies and input from stakeholders Systems Engineering, Test & Evaluation Conference, Sydney, Australia, October 2002 with hidden agendas weakens the effectiveness of the approach. The use of business and non-functional issues like cost and value is encouraging. However, using more than a few non-functional issues overburdens the effort required to get the stakeholders to rate the requirements based on each one of these non-functional issues. Wiegers Prioritisation Matrix Wiegers [10] describes a semi-quantitative analytical approach to requirements prioritisation. The approach distributes a set of estimated priorities across a continuum, rather than grouping them into just a few priority levels. This prioritisation scheme borrows from the QFD concept of customer value depending on both the customer benefit provided if a specific product feature is present and the penalty paid if that feature is absent. A feature’s attractiveness is directly proportional to the value it provides and inversely proportional to its cost and the technical risk associated with implementing it. All other things being equal, those features that have the highest risk-adjusted value/cost ratio should have the highest priority. The approach suffers from similar issues that AHP and QFD do not adequately handle. The notion of checking the subjective input, handling dependencies, and catering for requirements volatility is lacking. The stakeholders are restricted to using an ordinal 1-9 scale, and the approach only incorporates a limited number of nonfunctional and business issues in its prioritisation technique. Feature Relative Benefit Relative Penalty Total Value 1. Query status of a vendor order 5 3 13 2. Generate a Chemical Stockroom inventory